chore: add script to fix invalid proposal's scores_by_strategy
#574
Conversation
wa0x6e
changed the title
scores_by_strategy
There was a problem hiding this comment.
Pull Request Overview
This PR adds a cleanup script to fix invalid scores_by_strategy data format in proposals with weighted and quadratic voting types. The issue affects approximately 2300 proposals created over 3 years ago where the data has an extra array depth level ([[[0]]] instead of [[0]]).
- Adds a new script that identifies and fixes malformed JSON data in the
scores_by_strategycolumn - Provides both preview and run modes for safe execution
- Includes comprehensive logging and error handling for the data migration process
|
|
||
| if (isBuggy) { | ||
| // Fix the data by flattening: [[[0.75],[0]]] becomes [[0.75,0]] | ||
| const fixed = scoresByStrategy.map(item => item.flat()); |
There was a problem hiding this comment.
Using flat() without a depth parameter only flattens one level deep. If the nested structure is deeper than expected (e.g., [[[[0]]]]), this won't fully fix the issue. Consider using flat(Infinity) or specify the expected depth.
| const fixed = scoresByStrategy.map(item => item.flat()); | |
| const fixed = scoresByStrategy.map(item => item.flat(Infinity)); |
Copilot uses AI. Check for mistakes.
| type: proposal.type, | ||
| original: scoresByStrategy, | ||
| fixed: fixed | ||
| }); |
There was a problem hiding this comment.
The buggy data detection logic assumes all sub-items should be arrays, but this may incorrectly flag valid mixed data structures. Consider adding validation that the flattened result produces the expected format before marking as buggy.
| }); | |
| let fixed: any = null; | |
| if (looksBuggy) { | |
| // Fix the data by flattening: [[[0.75],[0]]] becomes [[0.75,0]] | |
| fixed = scoresByStrategy.map(item => item.flat()); | |
| // Only mark as buggy if the fixed data is in the expected format | |
| if (isArrayOfArraysOfNumbers(fixed)) { | |
| fixCandidates.push({ | |
| id: proposal.id, | |
| type: proposal.type, | |
| original: scoresByStrategy, | |
| fixed: fixed | |
| }); | |
| } |
Copilot uses AI. Check for mistakes.
Co-authored-by: Copilot <[email protected]>
Fix https://github.com/snapshot-labs/workflow/issues/620
See discussion in https://discord.com/channels/955773041898573854/1400932129378013255
This PR will add a script to cleanup the invalid format in the
scores_by_strategycolumn, for some proposals using weighted and quadratic voting type.There are ~2300 concerned proposals, all created more than 3 years ago (recent proposals don't have this issue)
The invalid format is
[[[0]]]instead of[[0]](there is one extra array depth), which is invalid according to the testsThe scripts had mostly been written by claude, but tested and fixed by an human.
You can find all the proposals with invalid scores with
Run
Run a preview to show only the buggy data (no update)
Run the script to fix the data